Convolutional Neural Networks for Object Classi cation in CUDA

f(x) = 1 1 + e−x , which is plotted in Figure 1. This is by far the most commonly used function. It has the nice property that its output is e ectively linear in the input if the size of the input is small. This means that neural networks with small weights essentially compute a linear function, and gradually increasing the weights allows one to control the degree of nonlinearity. The degree of nonlinearity controls the capacity of the neural network. For example, a network with purely linear neurons can only compute linear functions.